<!-- HTML header for doxygen 1.8.6-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<title>OpenCV: Processing images causing optical illusions</title>
<link href="../../opencv.ico" rel="shortcut icon" type="image/x-icon" />
<link href="../../tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../jquery.js"></script>
<script type="text/javascript" src="../../dynsections.js"></script>
<script type="text/javascript" src="../../tutorial-utils.js"></script>
<link href="../../search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="../../search/searchdata.js"></script>
<script type="text/javascript" src="../../search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
//<![CDATA[
MathJax.Hub.Config(
{
  TeX: {
      Macros: {
          matTT: [ "\\[ \\left|\\begin{array}{ccc} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{array}\\right| \\]", 9],
          fork: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ \\end{array} \\right.", 4],
          forkthree: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ \\end{array} \\right.", 6],
          forkfour: ["\\left\\{ \\begin{array}{l l} #1 & \\mbox{#2}\\\\ #3 & \\mbox{#4}\\\\ #5 & \\mbox{#6}\\\\ #7 & \\mbox{#8}\\\\ \\end{array} \\right.", 8],
          vecthree: ["\\begin{bmatrix} #1\\\\ #2\\\\ #3 \\end{bmatrix}", 3],
          vecthreethree: ["\\begin{bmatrix} #1 & #2 & #3\\\\ #4 & #5 & #6\\\\ #7 & #8 & #9 \\end{bmatrix}", 9],
          cameramatrix: ["#1 = \\begin{bmatrix} f_x & 0 & c_x\\\\ 0 & f_y & c_y\\\\ 0 & 0 & 1 \\end{bmatrix}", 1],
          distcoeffs: ["(k_1, k_2, p_1, p_2[, k_3[, k_4, k_5, k_6 [, s_1, s_2, s_3, s_4[, \\tau_x, \\tau_y]]]]) \\text{ of 4, 5, 8, 12 or 14 elements}"],
          distcoeffsfisheye: ["(k_1, k_2, k_3, k_4)"],
          hdotsfor: ["\\dots", 1],
          mathbbm: ["\\mathbb{#1}", 1],
          bordermatrix: ["\\matrix{#1}", 1]
      }
  }
}
);
//]]>
</script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js"></script>
<link href="../../doxygen.css" rel="stylesheet" type="text/css" />
<link href="../../stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<!--#include virtual="/google-search.html"-->
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="../../opencv-logo-small.png"/></td>
  <td style="padding-left: 0.5em;">
   <div id="projectname">OpenCV
   &#160;<span id="projectnumber">4.5.2</span>
   </div>
   <div id="projectbrief">Open Source Computer Vision</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "../../search",false,'Search');
</script>
<script type="text/javascript" src="../../menudata.js"></script>
<script type="text/javascript" src="../../menu.js"></script>
<script type="text/javascript">
$(function() {
  initMenu('../../',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="../../d3/d81/tutorial_contrib_root.html">Tutorials for contrib modules</a></li><li class="navelem"><a class="el" href="../../df/dee/tutorial_table_of_content_retina.html">Discovering the human retina and its use for image processing</a></li>  </ul>
</div>
</div><!-- top -->
<div class="header">
  <div class="headertitle">
<div class="title">Processing images causing optical illusions </div>  </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h2>Goal </h2>
<p>I will show here how the bioinspired module can reproduce a well-known optical illusion that our eyes perceive in certain light condition: The Adelson checkerboard.</p>
<h2>The Adelson checkerboard </h2>
<p>Looking at the checkerboard image below, human eyes perceive the "B" square lighter than the "A" square, although they are pictured in the very same RGB color. Of course in the physical world, checkerboard has a "B" square which is lighter than "A", but in this image the shadow of the green cylinder casting over the "B" square ends up in making the "A" and "B" squares actually having the same luminance.</p>
<div class="image">
<img src="../../checkershadow_illusion4med.jpg" alt="checkershadow_illusion4med.jpg"/>
<div class="caption">
Adelson checkerboard</div></div>
<p> Our visual system does "compensate" for the shadow, making us perceive the "B" square lighter, as the shadow wouldn't be there. This is due to local adaptation process that is performed in the foveal area.</p>
<p>You may find the original Adelson's explanation <a href="http://web.mit.edu/persci/people/adelson/checkershadow_description.html">here</a>.</p>
<p>Proof: You can convince yourself by using an image manipulation program, cutting out a portion of the two squares, and looking at them without any background. You can also measure the RGB values of the two squares with the picker tool.</p>
<p>In this image I've cropped a little piece of the A and B squares and I've put them side-by-side. It should be quite evident they have the same luminance. </p><div class="image">
<img src="../../checkershadow_illusion4med_proof.jpg" alt="checkershadow_illusion4med_proof.jpg"/>
<div class="caption">
Adelson checkerboard proof</div></div>
<p> It's worth to know that this illusion works because the checkerboard image, as you may see it on your laptop, casts on your retina with dimensions that cause the retina local adaptation to take into account both the two squares at the same time.</p>
<p>The foveal vision area is something like one inch at one meter (and because your eye moves continuously, with the so called "saccades", your brain is able to reconstruct the entire color scene in real time). This means that one single letter, either A or B, can hit your fovea at any time.</p>
<p>The point is that, even if you can't see both letters at the same time in a single eye fixation, when looking at one letter your fovea also takes into account light information from what is around it. This means that the fovea actually perceives also the neighboring cells.</p>
<p>The net effect is that when looking at one area, your eye locally adapts to luminance, filters noise, enforces contours, etc. considering what <em>surrounds</em> this area, and this makes the illusion work. We say that <em>the retina works in a "center surround" manner</em>.</p>
<p>So, the "A" cell being surrounded by lighter cells can be perceived darker. As a comparison, cell "B" 's neighborhood is darker and the cell "B" is then perceived lighter.</p>
<p>Finally, since shadow edges are soft, retina eliminates this information. Then shadows do not disrupt the overall chessboard observation making possible to "confidently being fooled" by the perceived cells luminance.</p>
<h2>Reproducing the illusion </h2>
<p>The bioinspired module does mimic (also) the parvocellular retina process, that is our foveal vision, and it does reproduce our eyes' local adaptation.</p>
<p>This means we can expect the parvo channel output to really contain luminance values similar to those we perceive with our eyes. Specifically, in this case we expect the "B" square RGB values to be actually lighter than the "A" ones.</p>
<p>To correctly mimic what our eye does we need opencv to do the local adaptation on the right image portion. This means we have to ensure that the opencv's notion of "local" does match with our image's dimensions, otherwise the local adaptation wouldn't work as expected.</p>
<p>For this reason we may have to adjust the <b>hcellsSpatialConstant</b> parameter (that technically specifies the low spatial cut frequency, or slow luminance changes sensitivity) depending by the image resolution.</p>
<p>For the image in this tutorial, the default retina parameters should be fine.</p>
<p>In order to feed the image to the bioinspired module, you can use either your own code or the <em>example_bioinspired_retinaDemo</em> example that comes with the bioinspired module.</p>
<p>Running </p><div class="fragment"><div class="line">example_bioinspired_retinaDemo -image checkershadow_illusion4med.jpg</div></div><!-- fragment --><p>will cause our image to be processed in both parvocellular and magnocellular channels (we are interested just in the first one).</p>
<p>If you choose to use your own code, please note that the parvocellular (and magnocellular) channel does require some iterations (frames to be processed) before actually getting steady.</p>
<p>Actually parvo (and magno) channel do cares about temporal information. That is, when you start feeding frames, it is similar to you with closed eyes; then you open them and you see the chessboard.</p>
<p>This is a static image but your retina just starts moving to a new context (eyes opening) and has to adapt.</p>
<p>While in this transient state the luminance information do matters, and you see more or less the absolute luminance values. Absolute luminance is exactly what you need <b>not</b> to look at in order to reproduce the illusion..</p>
<p>As soon as steady state is reached, you receive more contextual luminance information. Your eyes work in a center-surround manner and take into account the neighborhood luminance to evaluate the region of interest luminance level. And that's when our illusion comes out !</p>
<p>This is something that you don't need to worry about when you process videos, because you are naturally feeding the virtual retina with several frames, but you have to take care of it in order to process a single frame.</p>
<p>What you will actually need to do when processing a single frame, and you only need steady state response, is to repeatedly feed the retina with the same frame (this is what the example code does), as you would do with a still video. Alternatively you can set retina temporal parameters to 0 to get steady state immediately (<b>photoreceptorsTemporalConstant</b> and <b>hcellsTemporalConstant</b> parameters of the xml file); however in this case you should be aware that you are now making experiments with something that is deliberately less accurate in reproducing the behaviour of a real retina!</p>
<p>Here there is a small fragment of python code we used to process the image. It does 20 iterations. This is an arbitrary number that we found experimentally to be (more than) enough</p>
<div class="fragment"><div class="line"><span class="keyword">import</span> cv2 <span class="keyword">as</span> cv</div><div class="line"></div><div class="line">inputImage = <a class="code" href="../../d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56">cv.imread</a>(<span class="stringliteral">&#39;checkershadow_illusion4med.jpg&#39;</span>, 1)</div><div class="line">retina = cv.bioinspired.createRetina((inputImage.shape[1], inputImage.shape[0]))</div><div class="line"></div><div class="line"><span class="comment"># the retina object is created with default parameters. If you want to read</span></div><div class="line"><span class="comment"># the parameters from an external XML file, uncomment the next line</span></div><div class="line"><span class="comment">#retina.setup(&#39;MyRetinaParameters.xml&#39;)</span></div><div class="line"></div><div class="line"><span class="comment"># feed the retina with several frames, in order to reach &#39;steady&#39; state</span></div><div class="line"><span class="keywordflow">for</span> i <span class="keywordflow">in</span> range(20):</div><div class="line">    retina.run(inputImage)</div><div class="line"></div><div class="line"><span class="comment"># get our processed image :)</span></div><div class="line">retinaOut_parvo = retina.getParvo()</div><div class="line"></div><div class="line"><span class="comment"># show both the original image and the processed one</span></div><div class="line"><a class="code" href="../../df/d24/group__highgui__opengl.html#gaae7e90aa3415c68dba22a5ff2cefc25d">cv.imshow</a>(<span class="stringliteral">&#39;image&#39;</span>, inputImage)</div><div class="line"><a class="code" href="../../df/d24/group__highgui__opengl.html#gaae7e90aa3415c68dba22a5ff2cefc25d">cv.imshow</a>(<span class="stringliteral">&#39;retina parvo out&#39;</span>, retinaOut_parvo)</div><div class="line"></div><div class="line"><span class="comment"># wait for a key to be pressed and exit</span></div><div class="line"><a class="code" href="../../d7/dfc/group__highgui.html#ga5628525ad33f52eab17feebcfba38bd7">cv.waitKey</a>(0)</div><div class="line"><a class="code" href="../../d7/dfc/group__highgui.html#ga6b7fc1c1a8960438156912027b38f481">cv.destroyAllWindows</a>()</div><div class="line"></div><div class="line"><span class="comment"># write the output image on a file</span></div><div class="line"><a class="code" href="../../d4/da8/group__imgcodecs.html#gabbc7ef1aa2edfaa87772f1202d67e0ce">cv.imwrite</a>(<span class="stringliteral">&#39;checkershadow_parvo.png&#39;</span>, retinaOut_parvo)</div></div><!-- fragment --><p>Whatever method you used to process the image, you should end up with something like this:</p>
<div class="image">
<img src="../../checkershadow_parvo.png" alt="checkershadow_parvo.png"/>
<div class="caption">
Parvo output for adelson checkerboard</div></div>
 <h2>Analyzing the results </h2>
<p>We expected that the "B" pixels in the parvo channel output are lighter than "A" ones.</p>
<p>.. And in fact that is!</p>
<p>Looking at the resulting image might not tell us so much at a first glance: the "B" square looks lighter than "A" to our eyes, as it did in the input image. The difference is that, contrarily to the input image, now the RGB values of the pixels are actually lighter; note that when looking at the output image, we are actually applying the parvocellular process two times: first in the bioinspired module, then in our eyes. We can convince ourselves that the illusion appeared in the computed image by measuring the squares' luminance with the image manipulation program and the picker tool, or by cropping pieces of the squares and putting them side-by-side.</p>
<p>In the following image I cropped a portion of square "A" and a portion of square "B", and I placed them side-by-side, as I did for the original Adelson image.</p>
<div class="image">
<img src="../../checkershadow_parvo_proof.png" alt="checkershadow_parvo_proof.png"/>
<div class="caption">
Illusion reproduced</div></div>
<p> It should be quite evident that the "B" square is really lighter than the "A" square! Congratulations: you have just reproduced the Adelson illusion with the Bioinspired module!</p>
<h2>Credits </h2>
<p>I want to thank:</p>
<p><b>Alexandre Benoit</b> - for being so kind of explaining me how this whole thing works, for giving me the opportunity of writing this tutorial, and for reviewing it.</p>
<p><b>Edward Adelson</b> - for allowing me to freely use his checkerboard image.</p>
<p><b>Antonio Cuni</b> - for reviewing this tutorial and for writing the Python code. </p>
</div></div><!-- contents -->
<!-- HTML footer for doxygen 1.8.6-->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Fri Apr 2 2021 11:36:38 for OpenCV by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="../../doxygen.png" alt="doxygen"/>
</a> 1.8.13
</small></address>
<script type="text/javascript">
//<![CDATA[
addTutorialsButtons();
//]]>
</script>
</body>
</html>
